In the information age, databases are a precious commodity, storing immense quantities of data for use in various applications. Latency, or time needed to access stored database data, is a crucial metric for many performance-intensive applications. Portfolio management applications, for example, are generally performance-intensive.
In-memory databases are the fastest possible databases. In such databases, which place the dataset in main memory, any piece of information is available with almost zero latency. The memory requirements of such databases increase with the size of the stored dataset. Therefore, such databases become excessively expensive from a hardware perspective when datasets are very large. In addition, computer manufacturers limit the amount of memory that can be installed in their machines, which limits the maximum size of the dataset that can be stored.
Some database systems address this memory problem by using software to cache portions of the dataset in main memory while keeping the majority in secondary memory (i.e., secondary storage), such as on disk. While this approach solves one problem, it creates another: Complex software must keep track of the location of the objects being stored, moving copies of the in-memory objects back and forth from the disk. This approach also increases complexity and latency, as software must determine where to look for the object—in memory or on disk. In addition, desired data must be copied to the application's memory space because, for data integrity and functional reasons, users cannot be allowed direct access to the database copy of the object, whether it is found in the memory cache or on the disk.
By transparently mapping data into and out of main memory, instead of copying the data into main memory, data can be accessed at speeds close to those achievable if the data were in main memory without requiring the copying of the data into and out of main memory. This approach is exemplified by the system described in U.S. Pat. No. 7,222,117 by McGrogan, entitled “Segmented Global Area Database”, the entirety of which is herein incorporated by reference.
When dealing with additional sets of data, conventional database systems maintain a unified set of tables, each containing one type of database object. The tables are joined by linkages implemented in software inside the database, and are stored in the database as persistent data. Multiple databases cannot be quickly attached to and detached from each other, as they first need to be imported into the database as part of the unified set of tables. Creating additional sets of data by aggregating data already in the database increases the size of the database, as the amount of persistent data increases.